Adapting Transformer Networks for Document Summarization and Sentiment Analysis

Due to the increasing number of text-based content sources, demand for effective sentiment analysis and document summarization techniques has been increasing. Several transformer-based models, including “ELECTRA, BERT, XLNet, RoBERTa, DistilBERT, ALBERT” have emerged as promising alternatives traditional methods. This paper aims study effectiveness different transformer models performing on Yelp dataset. The analyze various models' performance tasks, identify their weaknesses, suggest possible improvements. It also thoroughly studies dataset, which over 5 million reviews. introduces that are used We then perform evaluation these using metrics measure performance. Some include ROUGE, F1-score, AUC-ROC, accuracy. According paper's experimental results, RoBERTa BERT better than other when it comes summarization. In addition, we identified weaknesses strengths each model. implementing domain-specific training fine-tuning improve results experiment revealed ones found strengths, contributes literature related use in documents by providing an extensive suggests modifications capabilities.